Display device and control method thereof

ABSTRACT

A display device and method for controlling the display device are discussed. In one embodiment, the display device includes a display unit configured to display an object; a tactile feedback unit configured to generate tactile feedback; and a controller configured to: assign first tactile feedback to a first region corresponding to a first object and second feedback to a second region corresponding to a second object; detect a first control input in the first region corresponding to the first object, the first control input controlling the first object displayed on the display unit; generate the first tactile feedback according to properties of the first object; and maintain the first tactile feedback when the first control input is moving from the first region to the second region corresponding to the second object while the first object is controlled by the first control input.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of co-pending U.S. patent applicationSer. No. 13/857,831 filed on Apr. 5, 2013, which is aContinuation-in-Part of U.S. application Ser. No. 13/679,582 filed onNov. 16, 2012 (now U.S. Pat. No. 8,451,248), which claims priority toApplication No. 10-2012-0108823, filed in the Republic of Korea on Sep.28, 2012. The entire contents of all of the above applications arehereby expressly incorporated by reference into the present application.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present specification relates to a display device, and moreparticularly to a display device capable of providing a user withtactile feedback using a tactile user interface and a control methodthereof.

2. Discussion of the Related Art

A display device may provide a user with a user interface for control ofan electronic appliance. Through use of the display device, the user cancontrol the electronic appliance by visually confirming an operationalstate of the electronic appliance. A recent display device is capable ofreceiving a control command of a user by detecting a user touch gesturevia a touch sensitive function. This allows the user to control anelectronic appliance by directly touching the digital device withoutusing input tools, such as a keyboard, mouse, etc. Such a display devicemay provide the user with a variety of user interfaces, examples ofwhich may include a visual user interface and a tactile user interface.A conventional tactile user interface usable with the display devicefunctions to generate only constant tactile feedback regardless of acontrol operation of the user, and thus has difficulty in adaptivelyresponding to the control operation of the user.

SUMMARY OF THE INVENTION

Accordingly, the present specification is directed to a display deviceand a control method thereof that substantially obviate one or moreproblems due to limitations and disadvantages of the related art.

An object of the present specification is to provide a display devicewhich provides a tactile user interface capable of adaptively generatingtactile feedback in response to a control operation of a user.

Additional advantages, objects, and features of the specification willbe set forth in part in the description which follows and in part willbecome apparent to those having ordinary skill in the art uponexamination of the following or may be learned from practice of thespecification. The objectives and other advantages of the specificationmay be realized and attained by the structure particularly pointed outin the written description and claims hereof as well as the appendeddrawings.

To achieve these objects and other advantages and in accordance with thepurpose of the specification, as embodied and broadly described herein,a display device includes a touch sensitive display unit configured todisplay an object, a tactile feedback unit configured to generatetactile feedback, and a controller configured to control the otherunits, wherein the controller is configured to detect a first touch thatis in contact with a region of a first object displayed on the displayunit, detect a second touch controlling the first object that isconsecutive to the first touch, control the first object in response tothe second touch, generate first tactile feedback according toproperties of the first object in response to the second touch in acontrolled state of the first object, detect a third touch controllingthe first object that is consecutive to the second touch, wherein thethird touch is detected in a region of a second object displayed on thedisplay unit, and generate the first tactile feedback in response to thethird touch in a controlled state of the first object.

In accordance with another aspect of the present specification, a methodof controlling a touch sensitive display device, includes detecting afirst touch of touching a region of a first object displayed on thedisplay unit, detecting a second touch controlling the first object thatis consecutive to the first touch, controlling the first object inresponse to the second touch, generating first tactile feedbackaccording to properties of the first object in response to the secondtouch in a controlled state of the first object, detecting a third touchcontrolling the first object that is consecutive to the second touch,wherein the third touch is detected in a region of a second objectdisplayed on the display unit, and generating the first tactile feedbackin response to the third touch in a controlled state of the firstobject.

It is to be understood that both the foregoing general description andthe following detailed description of the present specification areexemplary and explanatory and are intended to provide furtherexplanation of the specification as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a furtherunderstanding of the specification and are incorporated in andconstitute a part of this application, illustrate embodiment(s) of thespecification and together with the description serve to explain theprinciple of the specification. In the drawings:

FIG. 1 is a view illustrating a visual user interface and a tactile userinterface according to an embodiment of the present specification;

FIG. 2 is a view illustrating movement of a tactile feedback regiondepending on movement of an object according to an embodiment of thepresent specification;

FIG. 3 is a view illustrating a method of locking a movement directiondepending on movement of an object according to an embodiment of thepresent specification;

FIG. 4 is a view illustrating a method of locking tactile feedbackdepending on movement of an object according to an embodiment of thepresent specification;

FIG. 5 is a block diagram illustrating a display device according to anembodiment of the present specification;

FIG. 6 is a view illustrating a method of providing continuous tactilefeedback in response to a touch gesture controlling an object accordingto an embodiment of the present specification;

FIG. 7 is a view illustrating a method of providing continuous tactilefeedback in response to a touch gesture controlling an object accordingto another embodiment of the present specification;

FIG. 8 is a view illustrating a method of locking tactile feedbackdepending on movement of an object or sub-object according to anembodiment of the present specification;

FIG. 9 is a view illustrating a method of providing remote tactilefeedback in response to a non-contacting control input controlling anobject according to another embodiment of the present specification; and

FIG. 10 is a view illustrating a method of providing continuous tactilefeedback in response to a control input controlling an object accordingto another embodiment of the present specification.

DETAILED DESCRIPTION OF THE INVENTION

Hereinafter, the embodiments of the present specification will bedescribed in detail with reference to illustrations and descriptions ofthe accompanying drawings, but the present specification is not limitedor restricted by these embodiments.

Although the terms used in the following description are selected, asmuch as possible, from general terms that are widely used at presentwhile taking into consideration the functions obtained in accordancewith the present specification, these terms may be replaced by otherterms based on intensions of those skilled in the art, customs,emergence of new technologies, or the like. Also, in a particular case,terms that are arbitrarily selected by the applicant of the presentspecification may be used. In this case, the meanings of these terms maybe described in corresponding description parts of the specification.Accordingly, it should be noted that the terms used herein should beconstrued based on practical meanings thereof and the whole content ofthis specification, rather than being simply construed based on names ofthe terms.

In the present specification, a display device may be an electronicappliance that includes a display unit and a tactile feedback unit andprovides a user with a visual user interface and a tactile userinterface. The display device may include an electronic appliance havinga display unit, such as a smart phone, Personal Digital Assistant (PDA),cellular phone, MP3 music player, laptop computer, smart pad, tabletPersonal Computer (PC), television, and smart table, for example. Thedisplay device according to the present specification may transmitinformation to the user using the visual user interface and the tactileuser interface. The visual user interface may provide the user withvisual feedback using the display unit of the display device. The usercan visually confirm information provided from the display device, andcontrol the display device. The tactile user interface may provide theuser with tactile feedback using the tactile feedback unit of thedisplay device. The user can confirm information provided from thedisplay device via particular parts of the body that can perceivetactile sensation, such as the finger, etc., and control the displaydevice. The visual user interface and the tactile user interface may beseparately provided, or may be simultaneously provided.

FIG. 1 is a view illustrating a visual user interface and a tactile userinterface according to an embodiment of the present specification. Adigital device 10 may provide a visual user interface by displaying anobject on a touch sensitive display unit. The display device 10 maydisplay a plurality of objects including a first object 11 and a secondobject 12. The user can visually confirm the displayed objects, andcontrol the corresponding objects by touching the touch sensitivedisplay unit. The display device 10 may provide a tactile userinterface, which generates tactile feedback with respect to the visualuser interface. The tactile user interface may be generated in a regionwhere the first object 11 and the second object 12 are displayed. Thedisplay device 10 may generate first tactile feedback 13 in a regionwhere the first object 11 is displayed according to properties of thefirst object 11. Likewise, the display device 10 may generate secondtactile feedback 14 in a region where the second object 12 is displayedaccording to properties of the second object 12. Here, properties of theobject may include at least one of color, texture, weight, pattern, andbrightness of the displayed object. The display device 10 may generatedifferent tactile feedback with respect to respective objects havingdifferent properties. For example, in FIG. 1, assuming that the firstobject 11 and the second object 12 have different textures, the firsttactile feedback 13 and the second tactile feedback 14 may be differentfrom each other.

The display device 10 may additionally generate tactile feedbackcorresponding to friction between the displayed first object 11 and theuser's finger. Assuming that a background image is displayed and thefirst object 11 is displayed on the background image, the display device10 may generate the first tactile feedback 13 according to properties ofthe first object 11 and properties of the background image.

The display device 10 may detect pressure of a user touch gesture oftouching and pushing the display unit. The display device 10 may controlthe intensity of tactile feedback in proportion to the pressure of thetouch gesture. As such, the user can perceive that the user touches thedisplay unit with how strong pressure. The display device 10 may detectthe pressure of the touch gesture using a piezoelectric sensor or basedon a detection area of the touch gesture.

FIG. 2 is a view illustrating movement of a tactile feedback regiondepending on movement of an object according to an embodiment of thepresent specification. If a displayed object is moved by a user touchgesture, the display device may simultaneously move a tactile feedbackregion that is generated with respect to the corresponding objectaccording to movement of the object. As illustrated in the upper side ofFIG. 2, the display device 10 may display the first object 11 and thesecond object 12 as visual user interfaces. Additionally, the displaydevice 10 may generate the first tactile feedback 13 with respect to thefirst object 11 and the second tactile feedback 14 with respect to thesecond object 12, to provide the user with tactile user interfaces. Eachtactile feedback may be generated in a region where the correspondingobject is displayed. As illustrated in the lower side of FIG. 2, if theuser touches and drags the first object 11 to a position as representedby reference numeral 20, the display device 10 may move the first object11 so as to display the first object 11 at the position 20. The displaydevice 10 may also move the first tactile feedback 13 with respect tothe first object 11 simultaneously with movement of the first object 11.In this case, the first tactile feedback 13 may be generated in a regionwhere the first object 11 is displayed.

The display device 10 may generate the first tactile feedback 13according to properties of the first object 11. Here, properties of thefirst object 11, as described above with reference to FIG. 1, mayinclude texture, pattern, and color, for example. In addition to thefirst tactile feedback 13, the display device 10 may further generatetactile feedback corresponding to friction between the displayed firstobject 11 and the user's finger. Assuming that a background image isdisplayed and the first object 11 is controlled on the background image,the display device 10 may further generate tactile feedback according toproperties of the first object 11 and properties of the backgroundimage, in addition to the first tactile feedback 13.

The second object 12, which is not subjected to the above describedtouch and drag operation of the user, may be displayed at the sameposition without movement. Thereby, the second tactile feedback 14 withrespect to the second object 12 may be generated in a region where thesecond object 12 is displayed without movement. In this way, the displaydevice 10 may provide the user with continuous tactile feedback withrespect to an object selected by the user even if the user moves thecorresponding object.

FIG. 3 is a view illustrating a method of locking a movement directiondepending on movement of an object according to an embodiment of thepresent specification. The display device 10 may lock a particularmovement direction of an object when moving the object in response to auser touch gesture. The user touch gesture may be divided into an X-axiscomponent and a Y-axis component on the touch sensitive display unit.When the user touch gesture of moving the object contains a plurality ofmovement direction components, the display device 10 may move the objectby extracting only a main movement direction component of the touchgesture. More specifically, as illustrated in FIG. 3, even if it isdetected that a touch gesture 20 of moving the first object 11 has anX-axis movement direction component and a Y-axis movement directioncomponent, the display device 10 may extract only the Y-axis movementdirection component of the touch gesture 20. As a result, the displaydevice 10 may move the first object 11 along a Y-axis in a positivedirection without X-axis movement.

The display device 10 may perform locking of the particular movementdirection in consideration of properties of the object. For example, inthe case of a document or web-page that requires upward or downwardscrolling, the display device 10 may move the object upward or downwardusing only the Y-axis movement direction component even if the touchgesture 20 contains the X-axis movement direction component and theY-axis movement direction component. Likewise, in the case of a documentor web-page that requires leftward or rightward scrolling, the displaydevice 10 may move the object leftward or rightward using only theX-axis movement direction component even if the touch gesture 20contains the X-axis movement direction component and the Y-axis movementdirection component.

The display device 10 may generate tactile feedback in response to auser touch gesture of moving the object. The display device 10, asillustrated in the right side of FIG. 3, may generate tactile feedbackin response to the user touch gesture 20. When the user touch gesture 20is detected in a region of the first object 11, the display device 10may generate the first tactile feedback 13. The display device 10 maymove the region in which the first tactile feedback 13 is generatedsimultaneously with the first object 11 being moved by the touch gesture20. If the user touch gesture 20 of moving the first object 11 enters aregion in which the second object 12 is displayed, the display device 10may generate the second tactile feedback 14 in the region of the secondobject 12. If the first object 11 and the second object 12 havedifferent properties, the display device 10 may generate the secondtactile feedback 14 different from the first tactile feedback 13 in theregion of the second object 12. The user can perceive the second tactilefeedback 14 via tactile sensation, and recognize that the user touchgesture 20 deviates from the region of the first object 11 and enter theregion of the second object 12. In this way, the display device 10 mayindicate that the user touch gesture 20 is shifted from the region ofthe first object 11 to the region of the second object 12.

According to an embodiment, the above described second object 12 may bea background image displayed on the display unit of the display device10. Accordingly, even in the case in which the touch gesture 20 ofmoving the first object 11 deviates from the region in which the firstobject 11 is displayed, the display device 10 may generate tactilefeedback different from the first tactile feedback 13. Even in thiscase, the user can perceive the tactile feedback different from thefirst tactile feedback 13, and recognize that the user touch gesture 20deviates from the region of the first object 11.

FIG. 4 is a view illustrating a method of locking tactile feedbackdepending on movement of an object according to an embodiment of thepresent specification. The display device 10 may provide the user withcontinuous tactile feedback when controlling a displayed object inresponse to a user touch gesture. Specifically, the display device 10may generate initial first tactile feedback even if a user touch gestureof touching a first region in which the first tactile feedback isgenerated enters a second region in which second tactile feedback isgenerated while maintaining contact with the touch sensitive displayunit.

The display device 10, as illustrated in the left side of FIG. 4, maycontrol a displayed object in response to a user touch gesture. Controlof an object may include movement, rotation, enlargement and contractionof an object. Also, control of the object may include selection of anobject, or call of a subordinate menu with respect to an object. Thedisplay device 10 may generate continuous tactile feedback in responseto a user touch gesture controlling an object. The display device 10, asillustrated in the right side of FIG. 4, may generate tactile feedbackin response to the user touch gesture 20. The display device 10 maygenerate the first tactile feedback 13 when the user touch gesture 20 isdetected in the region of the first object 11.

As the first object 11 is moved by the touch gesture 20, the displaydevice 10 may simultaneously move a region in which the first tactilefeedback 13 is generated. The display device 10, as described above withreference to FIG. 3, may move the first object 11 by extracting only avertical movement component among movement direction components of thetouch gesture 20. As such, the user touch gesture 20 may deviate fromthe region of the first object 11 and enter the region of the secondobject 12.

If the touch gesture 20 of moving the first object 11 is detected in aregion in which the second object 12 is displayed, the display device 10may lock tactile feedback in response to the touch gesture 20 andgenerate the first tactile feedback 13 in the region of the secondobject 12. Even if the first object 11 and the second object 12 havedifferent properties, the display device 10 may generate the firsttactile feedback 13 instead of the second tactile feedback in the regionof the second object 12. That is, as illustrated in FIG. 2, even if thesecond tactile feedback 14 is generated in the region of the secondobject 12, the display device 10 may replace the second tactile feedback14 by the first tactile feedback 13 when the touch gesture 20controlling the first object 11 is detected in the region of the secondobject 12. In this way, while the user touch gesture 20 controls thefirst object 11, the display device 10 may lock tactile feedback withrespect to the user touch gesture 20, so as to generate the firsttactile feedback 13 regardless of the region in which the touch gesture20 is detected.

As described above, the display device 10 may indicate that an objectthat is under control of the user is still the first object 11 bycontinuously generating the first tactile feedback 13 regardless of theregion in which the touch gesture 20 is detected. The user can visuallyconfirm that the user touch gesture 20 touches the region of the secondobject 12, and recognize, via tactile sensation, that the user touchgesture 20 controls the first object 11 for generation of the firsttactile feedback 13.

In the case in which control of a touch gesture with respect to a firstobject ends, the display device 10 may unlock tactile feedback that hasbeen locked with respect to the corresponding touch gesture. The displaydevice 10 may detect motion that the user's hand leaves the touchsensitive display unit as the end of the touch gesture controlling thefirst object. As such, if the user touch gesture controlling the firstobject ends as the user's hand leaves the region of the second object 12and thereafter the user's hand again touches the region of the secondregion 12, the display device 10 may generate second tactile feedbackrather than first tactile feedback that has been locked in thecorresponding region. Note that the display device 10 may set a timethreshold. If the user again touches the region of the second objectwithin the time threshold after the user's hand leaves from the regionof the second object, the display device 10 may continuously locktactile feedback so as to generate the first tactile feedback in theregion of the second object.

The display device 10 may also unlock tactile feedback that has beenlocked in the case in which a distance between a position where a touchgesture is detected and a first object exceeds a preset distancethreshold. The display device 10 may unlock tactile feedback that hasbeen locked if the touch gesture controlling the first object isdetected in a region exceeding the distance threshold from the firstobject. Additionally, even in the case in which the user moves theuser's finger that is touching the first object to a position spacedapart from the first object by more than the distance threshold whilemaintaining contact with the display unit, the display device 10 mayunlock the tactile feedback that has been locked. Once the tactilefeedback has been unlocked, the display device 10 may generate tactilefeedback according to properties of the object displayed in the regionin which the touch gesture is detected.

According to an embodiment, the above described second object may be abackground image displayed on the display unit of the display device 10.Accordingly, the display device 10 may generate the first tactilefeedback even in the case in which the touch gesture of moving the firstobject deviates from the region in which the first object is displayed.Even in this case, the user may continuously perceive the first tactilefeedback and recognize that the user touch gesture continuously controlsthe first object although the touch gesture deviates from the region ofthe first object.

The display device 10, as described above, may lock tactile feedbackbetween a plurality of objects or between an object and a backgroundimage. The embodiment of the present specification may be applied evenin the case in which a plurality of feedback is generated within asingle object. For example, a first region and a second region forgenerating first tactile feedback and second tactile feedbackrespectively may be present within a first object. As described above,in the case in which a touch gesture of touching the first region entersthe second region while controlling the first region, the display device10 may lock tactile feedback in response to the corresponding touchgesture so as to continuously generate the first tactile feedback. Inthis way, the display device 10 may generate the first tactile feedbackin the second region in response to the touch gesture.

FIG. 5 is a block diagram illustrating a display device according to anembodiment of the present specification. In FIG. 5, the display devicemay include a sensor unit 101, a display unit 102, a tactile feedbackunit 103, and a controller 104.

The sensor unit 101 may transmit a user input or an environmentrecognized by the display device to the controller 104 using a pluralityof sensors equipped in the display device. The sensor unit 101 mayinclude a plurality of sensing means. In an embodiment, the plurality ofsensing means may include a piezoelectric sensor, a video sensor, and atouch sensor, for example. In the present specification, thepiezoelectric sensor and the touch sensor may detect a user toughgesture. The display device may determine a position on the display unitwhere the touch gesture is detected using the sensor unit 101. Thedisplay device may provide tactile feedback at a position or region inwhich the touch gesture is detected. The sensor unit 101 is a genericterm of the aforementioned various sensing means, and may detect touchgestures having various properties and transmit the sensed results tothe controller 104, so as to allow the display device to perform acorresponding operation. The aforementioned sensors may be embodied intoseparate elements included in the display device, or may be combined toconstitute at least one element included in the display device.

The display unit 102 may output an image on a display screen. Thedisplay unit 102 may serve as the above described touch sensor under theassumption that the display unit 102 is of a touch sensitive displaytype. Accordingly, the display unit 102 may detect a touch gesturethereto, and transmit the touch gesture to the controller 104. Thedisplay unit 102 may serve to display a visual user interface on adisplay panel, or to control display of an image. In the presentspecification, the display unit 102 may display a visual user interface,and detect a user touch gesture. Also, the display unit 102 may providethe user with tactile feedback in conjunction with the tactile feedbackunit 103 that will be described hereinafter. The display unit 102 mayinclude a flexible display.

The tactile feedback unit 103 may provide a tactile user interface. Thetactile feedback unit 103 may provide a tactile user interfacecorresponding to the visual user interface provided by the display unit102. The tactile feedback unit 103 may generate different tactilefeedback according to properties of a displayed object. The propertiesof the object may include at least one of color, texture, weight,pattern and brightness.

A method of providing the user with the tactile user interface using thetactile feedback unit 103 is as follows. Tactile feedback may beprovided with respect to the user's finger or a stylus that touches thedisplay unit 102 using a minutely vibrating actuator. The tactilefeedback unit 103 may adjust the frequency and magnitude of vibration,thereby adjusting friction between the user's finger and the displayunit 102. Additionally, the tactile feedback unit 103 may provide theuser with tactile feedback by applying a fine current to the displayunit 102. The tactile feedback unit 103 may adjust the strength andgeneration period of current, which may provide the user with differenttactile sensations. The tactile feedback unit 103 may provide tactilefeedback using ultrasonic resonance. The tactile feedback unit 103 mayprovide the user with tactile feedback by generating ultrasonic waveplural times and causing resonation of the ultrasonic waves within aspecific range. The tactile feedback unit 103 may provide differenttactile feedback by adjusting the magnitude of resonance frequency andthe resonance generation period of the frequency.

The controller 104 may control the above described units of the displaydevice 10, and manage data transmission/reception between the units. Inthe present specification, the controller 104 may determine whether ornot to lock tactile feedback in response to a touch gesture based onpositions where an object to be controlled by a user touch gesture andthe touch gesture are detected, or based on continuity of the touchgesture. For example, the controller 104 may provide first tactilefeedback in response to a touch gesture that is detected in the regionin which a first object is displayed. The first tactile feedback may begenerated according to properties of the first object. If the touchgesture is dragging the first object to a region where a second objector a background image is displayed while controlling the first object,the controller 104 may lock the tactile feedback in response to thecorresponding touch gesture. That is, even if the first object isdetected at the outside of a region in which the first object isinitially displayed while the touch gesture controls the first object,the controller 104 may generate the first tactile feedback in a regionin which the touch gesture is detected. The controller 104 maycontinuously lock the tactile feedback while the user's finger comesinto contact with the display unit via the touch gesture. The controller104 may unlock the tactile feedback with respect to the touch gesture ifcontact between the touch gesture and the display unit ends. In otherwords, unlocking of the tactile feedback may be permitted only when theuser's hand leaves the touch sensitive display unit 102.

In the present specification, the controller 104 may activate the touchsensitive display unit 102 to detect a user touch gesture. In this case,the touch sensitive display unit 102 may do not provide a visual userinterface, but provide a tactile user interface. Additionally, to detectthe user touch gesture, the controller 104 may activate a touch sensor,a piezoelectric sensor, or a pressure sensor of the sensor unit 101instead of the touch sensitive display unit 102.

FIG. 5 is a block diagram according to the embodiment of the presentspecification, and separately displayed blocks represent logicallydistinguished elements of the device. Accordingly, the elements of theabove described device may be mounted in a single chip or in a pluralityof chips according to a device design.

FIG. 6 is a view illustrating a method of providing continuous tactilefeedback in response to a touch gesture to control an object accordingto an embodiment of the present specification.

The display device may detect a first touch of touching a region of afirst object displayed on the display unit (S10). The first object maybe displayed on the display unit to assist the user in visuallyconfirming the first object. The user may touch the region in which thefirst object is displayed, in order to control the first object.

The display device may detect a second touch controlling the firstobject that is consecutive to the first touch (S20). The second touchmay be a consecutive touch gesture of the first touch. Here, theconsecutive touch gesture, as described above with reference to FIG. 4,may indicate a touch gesture that is input while contact between theuser's finger and the display unit is maintained, or a touch gesturedetected within a preset time threshold after a previous touch gestureends even if contact between the user's finger and the display unit isnot maintained. The display device may detect the second touchcontrolling the first object. The second touch may control the firstobject to move or rotate the first object. Additionally, the secondtouch may include motion to enlarge or contract the first object.

The display device may control the first object in response to thesecond touch (S30). The display device may control the first object inresponse to the detected second touch. The first object may be moved orrotated on the display unit by the second touch. Additionally, the firstobject may be enlarged or contracted by the second touch.

The display device may generate first tactile feedback according toproperties of the first object in response to the second touch in acontrolled state of the object (S40). The display device may generatethe first tactile feedback according to properties of the first objectusing the tactile feedback unit. The tactile feedback unit, as describedabove with reference to FIG. 5, may generate the first tactile feedbackin response to the second touch using ultrasonic vibration, fine currentor ultrasonic resonance. The user may recognize that the first object isunder control of the user by perceiving the first tactile feedback. Thatis, the display device may display that the first object is undercontrol using the display unit, and may generate the first tactilefeedback representing that the first object is under control using thetactile feedback unit. The display device may generate the first tactilefeedback according to properties of the first object. The display devicemay generate the first tactile feedback in consideration of at least oneof texture, pattern, shape, size, intensity and brightness of thedisplayed first object.

The display device may generate additional tactile feedback according toproperties of a background image. The display device may generate theadditional tactile feedback in consideration of properties of thebackground image and the first object together. For example, the displaydevice may generate not only first tactile feedback tactile feedback,but also additional tactile feedback corresponding to friction inconsideration of textures of the background image and the first object.

The display device may generate additional tactile feedbackcorresponding to friction between the first object and the user'sfinger. The display device may determine the magnitude of frictionbetween the first object and the user's finger according to texture ofthe first object, and generate additional tactile feedback in additionto the first tactile feedback.

The display device may detect a third touch controlling the first objectthat is consecutive to the second touch (S50). In this case, the thirdtouch may be detected in a region of a second object displayed on thedisplay unit. The user touch gesture, as described above with referenceto FIG. 4, may deviate from the region in which the first object isdisplayed and enter the region in which the second object is displayedwhile continuously controlling the first object. If the touch gesturecontrolling the first object is detected in the region in which thesecond object is displayed, the display device may recognize the touchgesture as a third touch. As described above with reference to FIG. 4,the third touch may be a touch gesture that is input that is consecutiveto the second touch in a state in which contact between the user'sfinger and the display unit is maintained. The third touch may indicatea touch gesture detected within a preset time threshold after the secondtouch ends although the user's finger does not continuously touch thedisplay unit.

The display device may generate the first tactile feedback in responseto the third touch in a controlled state of the object (S60). Thedisplay device, as described above with reference to FIG. 4, may locktactile feedback while an object is controlled. Accordingly, the displaydevice may generate the first tactile feedback in response to the thirdtouch controlling the first object. The display device may generate thefirst tactile feedback regardless of a region or position where thethird touch is detected while the first object is controlled by thethird touch. In an alternative embodiment, the display device maygenerate the first tactile feedback until contact between the user'sfinger and the display unit by the third touch is released.

The display device may unlock the first tactile feedback that has beenlocked if control of the first object by the third touch ends. In analternative embodiment, the display device may unlock the tactilefeedback that has been locked until contact between the user's fingerand the display unit by the third touch is released. The display devicemay unlock the tactile feedback that has been locked even when the thirdtouch is detected at a position exceeding a preset distance thresholdfrom the first object. Once the tactile feedback has been unlocked, thedisplay device may generate tactile feedback according to properties ofan object or a background image displayed in the region in which theuser touch gesture is detected.

FIG. 7 is a view illustrating a method of providing continuous tactilefeedback in response to a touch gesture to control an object accordingto another embodiment of the present specification.

The display device may detect a first touch of touching a region of afirst object displayed on the display unit (S110). The first object maybe displayed on the display unit to assist the user in visuallyconfirming the first object. The user may touch the region in which thefirst object is displayed, in order to control the first object.

The display device may detect a second touch controlling the firstobject that is consecutive to the first touch, and control the firstobject in response to the first touch (S120). The second touch may be aconsecutive touch gesture of the first touch. Here, the consecutivetouch gesture, as described above with reference to FIG. 4, may indicatea touch gesture that is input while contact between the user's fingerand the display unit is maintained, or a touch gesture detected within apreset time threshold after a previous touch gesture ends even ifcontact between the user's finger and display unit is not maintained.The display device may detect a second touch controlling the firstobject. The second touch may control the first object to move or rotatethe first object. Additionally, the second touch may include motion toenlarge or contract the first object.

The display device may control the first object in response to thedetected second touch. The first object may be moved or rotated on thedisplay unit by the second touch. Additionally, the first object may beenlarged or contracted by the second touch.

The display device may generate first tactile feedback according toproperties of the first object in response to the second touch in acontrolled state of the object (S130). The display device may generatethe first tactile feedback according to properties of the first objectusing the tactile feedback unit. The tactile feedback unit, as describedabove with reference to FIG. 5, may generate the first tactile feedbackin response to the second touch using ultrasonic vibration, fine currentor ultrasonic resonance. The user may recognize that the first object isunder control of the user by perceiving the first tactile feedback. Thatis, the display device may display that the first object is undercontrol using the display unit, and may generate the first tactilefeedback representing that the first object is under control using thetactile feedback unit. The display device may generate the first tactilefeedback according to properties of the first object. The display devicemay generate the first tactile feedback in consideration of at least oneof texture, pattern, shape, size, intensity and brightness of thedisplayed first object.

The display device may generate additional tactile feedback according toproperties of a background image. The display device may generate theadditional tactile feedback in consideration of properties of thebackground image and the first object. For example, in addition to thefirst tactile feedback, the display device may generate tactile feedbackcorresponding to friction in consideration of textures of the backgroundimage and the first object.

The display device may generate additional tactile feedbackcorresponding to friction between the first object and the user'sfinger. The display device may determine the magnitude of frictionbetween the first object and the user's finger according to texture ofthe first object to generate the tactile feedback in addition to thefirst tactile feedback.

The display device may judge whether or not the second touch deviatesfrom the region of the first object (S140). The user touch gesture, asdescribed above with reference to FIG. 4, may deviate from the region inwhich the first object is displayed while controlling the first object.The display device may generate the first tactile feedback if the secondtouch does not deviate from the region of the first object.

The display device may judge whether or not contact between the secondtouch and the display unit is released if the second touch deviates fromthe region of the first object (S150). The display device, as describedabove with reference to FIG. 4, may lock the first tactile feedback inresponse to the second touch while contact between the second touch andthe display unit is maintained. The display device may generate thefirst tactile feedback regardless of the region in which the secondtouch is detected if contact between the second touch and the displayunit is not released.

The display device may generate second tactile feedback if contactbetween the second touch and the display unit is released (S160). Asdescribed above with reference to FIG. 5, the second tactile feedbackmay be different from the first tactile feedback. The display device maygenerate the second tactile feedback according to properties of anobject or a background image located in a region where the second touchis detected. The display device may generate the second tactile feedbackin consideration of at least one of texture, pattern, shape, size,intensity and brightness of the object or the background image in theregion in which the displayed second touch is detected.

FIG. 8 is a view illustrating a method of locking tactile feedbackdepending on movement to an object according to an embodiment of thepresent specification. The display device may provide the user withcontinuous tactile feedback when controlling a displayed object inresponse to a user control input. Specifically, the display device maygenerate tactile feedback corresponding to an object to be controlled bya user control input. In addition, while the object is under control ofthe user, the display device may maintain the tactile feedback withrespect to the object that is under control of the user even if the usercontrol input is detected in a region where another object is displayed.

As illustrated in FIG. 8, the display device may control a displayedobject according to a user control input. Here, control of the objectmay include movement, rotation, enlargement, and contraction of theobject. In addition, control of the object may include selection of theobject, or call of a subordinate menu with respect to the object. Inthis way, the display device may generate continuous tactile feedback inresponse to the user control input controlling the corresponding object.

In FIG. 8, a first object 81 may include a first sub-object 82 and asecond sub-object 83. Here, the first object 81, the first sub-object82, and the second sub-object 83 may correspond to different tactilefeedback respectively. That is, the display device 10 may generate firsttactile feedback in a region where the first object 81 is displayed,second tactile feedback in a region where the first sub-object 82 isdisplayed, and third tactile feedback in a region where the secondsub-object 83 is displayed. In addition, the display device 10 maydisplay a second object 84, and may generate fourth tactile feedback ina region where the second object 84 is displayed. Accordingly, the usercan perceive different tactile feedback in the respective regions, andrecognize which of the respective regions is touched by the user. Thefirst sub-object 82 and the second sub-object 83 are included in thefirst object 81, and therefore may be controlled together when the usercontrols the first object 81.

In FIG. 8, the user may control the first object 81 by touching theregion where the first sub-object 82 is displayed. The display device 10may set the first object 81 so as to move upward or downward. If theuser moves a control input right upward, the display device 10 may movethe first object 81 upward. Accordingly, for example, as the usercontrol input is moved, the display device 10 may sequentially detectthe user control input in the region where the first sub-object 82 isdisplayed, in the region where the first object 81 is displayed, in theregion where the second sub-object 83 is displayed, in the region wherethe first object 81 is displayed, and in the region where the secondobject 84 is displayed. In addition, the display device 10 may generatetactile feedback corresponding to a position where the user controlinput is detected. That is, the display device may generate, in responseto the user control input, the second tactile feedback, the firsttactile feedback, the third tactile feedback, the first tactilefeedback, and the fourth tactile feedback in this sequence.

In another embodiment, the display device, as described above withreference to FIG. 4, may lock tactile feedback. The display device maygenerate tactile feedback corresponding to each region when a usercontrol input, which is moved within a region in which a first object isdisplayed, is detected in the corresponding region. That is, if the usercontrol input is sequentially detected in a region where a firstsub-object is displayed, in the region where the first object isdisplayed, and in a region where a second sub-object is displayed, thedisplay device may sequentially generate second tactile feedback, firsttactile feedback, and third tactile feedback in the respective regionswhere the user control input is detected. In other words, the displaydevice may generate tactile feedback corresponding to each region wherethe user control input is detected if the user control input is movedwithin the first object that is under control of the user.

The display device may lock tactile feedback if the user control inputis detected in a region where a second object is displayed while thefirst object is under control of the user. More specifically, thedisplay device may generate the tactile feedback corresponding to anyone sub-object included in the first object if the user control input isdetected in the region where the second object is detected while thefirst object is under control of the user. If the first object containsa plurality of sub-objects, the display device may generate tactilefeedback corresponding to the last sub-object that the user controlinput touches within the region where the first object is displayed. Forexample, in FIG. 8, the display device may generate the third tactilefeedback corresponding to the second sub-object if the user controlinput is detected in the region where the second object is displayed.This is because the second sub-object is the last sub-object that theuser control input touches within the region where the first object isdisplayed. The above described tactile feedback locking may be appliedin the case in which the user control input is detected in the regionwhere the second object is displayed at or after a preset time thresholdafter the user control unit is detected in the region where the firstobject is displayed.

In another embodiment, the display device may lock tactile feedback inresponse to a user control input using a time threshold. The displaydevice may lock tactile feedback if the user control input is detectedin a region where a second object is displayed while a first object isunder control of the user. More specifically, the display device maygenerate tactile feedback corresponding to any one sub-object includedin the first object if the user control input is detected in the regionwhere the second object is displayed. In FIG. 8, if the user controlinput is moved from a first sub-object to the second object within apreset time threshold, the display device may generate second tactilefeedback corresponding to the first sub-object in the region where thesecond object is displayed. That is, the display device may lock thesecond tactile feedback corresponding to the first sub-object that theuser control input initially touches within the first object in responseto the corresponding user control input. In this way, the display devicemay maintain the second tactile feedback corresponding to the firstsub-object even if the user control input is moved to and detected inthe region where the first object is displayed, in the region where thesecond sub-object is displayed, or in the region where the second objectis displayed, rather than in the region where the first sub-object isdisplayed, within the time threshold.

As described above, the display device may indicate an object that isbeing controlled by the user control input by continuously generatingtactile feedback corresponding to the object that is under control ofthe user. For example, the user may visually confirm that the usercontrol input touches the region where the second object is displayed,and may recognize that the first object is under control of the user viatactile sensation.

FIG. 9 is a view illustrating a method of generating non-contact tactilefeedback according to an embodiment of the present specification. Adisplay device, designated by reference numeral 91, may generatenon-contact tactile feedback by detecting a user control input. Here,the user control input may include user motion including hand motion,user views, and a control input using a control device. In addition, thedisplay device may transmit tactile feedback to the user using amagnetic field, compressed air, ultrasound wave, vibration, and wearabletactile device even if the user does not directly touch the displaydevice. The display device may include an airborne ultrasound tactiledisplay. Accordingly, the display device may transmit tactile feedbackby applying acoustic radiation pressure to the back of the user's handusing airborne ultrasound tactile feedback. In the followingdescription, an embodiment in which the display device perceives userhand motion and provides the user with tactile feedback using ultrasoundwave will be described.

The display device 91 may generate tactile feedback by detecting acontrol input of a user 90. The display device 91 may display a firstobject 92 and a second object 93, and may detect the control input ofthe user 90 controlling the first and second objects 92 and 93. If thefirst object 92 is under control of the user 90, i.e. if the user's hand20 is present in a first region 94 corresponding to the first object 92,the display device 91 may generate first tactile feedback in the firstregion 94. If the user's hand 20 is detected in a second region 95corresponding to the second object 93 while the first object 92 is undercontrol of the user, the display device may lock the first tactilefeedback. That is, the display device may generate the first tactilefeedback corresponding to the first object 92 in a region where theuser's hand 20 is located while the first object 92 is under control ofthe user 90. In this way, it is possible to provide the user with thefirst tactile feedback even if the user control input is moved to thesecond region 95 corresponding to the second object 93 while the firstobject 92 is under control of the user 90.

The display device may generate airborne ultrasound tactile feedback ata position where the user's hand 20 is located. If the user controlinput takes the form of user views or a control input using a controldevice, the display device may generate first tactile feedback in otherbody parts except for the user's hand or in the control device. Inaddition, the display device, as described above, may transfer tactilefeedback by ejecting compressed air to the user or by transmitting asignal to a tactile device.

FIG. 10 is a flowchart illustrating a method of generating non-contacttactile feedback according to an embodiment of the presentspecification. The display device may be operated upon receiving a usercontrol input. The user control input may include at least one of atouch input, a gesture input, a sound input, and a signal input. Thedisplay device may detect a first control input of selecting a firstobject displayed on the display unit (S210). The user may visuallyconfirm the first object because the first object is displayed on thedisplay unit. The user may select the first object in order to controlthe first object.

Next, the display device may detect a second control input controllingthe first object, which is consecutive to the first control input(S220), and may control the first object in response to the secondcontrol input (S230). The second control input may be a control inputconsecutive to the first control input. Here, the consecutive controlinput may refer to a next control input that is detected within a presettime threshold after a previous control input ends, as described abovewith reference to FIG. 4. In this way, the display device may detect thesecond control input controlling the first object. The second controlinput may serve to move or rotate the first object. In addition, thesecond control input may include enlargement or contraction of the firstobject. The display device may control the first object in response tothe detected second control input. Thereby, the first object may bemoved or rotated on the display unit in response to the second controlinput. In addition, the first object may be enlarged or contracted insize in response to the second control input.

Next, the display device may generate first tactile feedback accordingto properties of the first object in response to the second controlinput as the first object is under control of the user (S240). Thedisplay device may generate the first tactile feedback according toproperties of the first object using the tactile feedback unit. Thetactile feedback unit may generate the first tactile feedback inresponse to the second control input using ultrasound wave, compressedair, vibration, a wearable tactile device, or acoustic radiationpressure as described above with reference to FIGS. 5 and 9. The usermay recognize that the first object is under control of the user bysensing the first tactile feedback. That is, the display device maydisplay that the first object is under control of the user using thedisplay unit, and may generate the first tactile feedback representingthat the first object is under control of the user using the tactilefeedback unit. The display device may generate the first tactilefeedback according to properties of the first object. That is, thedisplay device may generate the first tactile feedback according to atleast one of texture, pattern, shape, size, intensity, and brightness ofthe displayed first object.

In addition, the display may generate additional tactile feedbackaccording to properties of a background image. The display device maygenerate additional tactile feedback in consideration of properties ofthe background image as well as properties of the first object. Forexample, in addition to the first tactile feedback, the display devicemay generate new tactile feedback corresponding to friction between thebackground image and the first object in consideration of texture of thebackground image and texture of the first object.

Next, the display device may detect a third control input controllingthe first object, which is consecutive to the second control input(S250). In this case, the third control input may be detected in aregion of the display unit where a second object is displayed. The usercontrol input may deviate from a first region corresponding to the firstobject and enter a second region corresponding to the second objectwhile the first object is under control of the user, as described abovewith reference to FIG. 9. The display device may detect the thirdcontrol input if the control input controlling the first object isdetected in the second region corresponding to the second object. Asdescribed above with reference to FIG. 4, the third control input mayrefer to a control input detected within a preset time threshold afterthe second control input ends.

Next, the display device may generate the first tactile feedback inresponse to the third control input as the first object is under controlof the user (S260). The tactile feedback unit may generate the firsttactile feedback in response to the third control input using at leastone of ultrasound wave, compressed air, vibration, a remote tactiledevice, a wearable tactile device, or acoustic radiation pressure asdescribed above with reference to FIGS. 5 and 9. The display device maylock the tactile feedback while the first object is under control of theuser, as described above with reference to FIG. 9. Accordingly, thedisplay device may generate the first tactile feedback in response tothe third control input controlling the first object. The display devicemay maintain the first tactile feedback in response to the third controlinput detected in the second region while the third control inputcontrols the first object.

The display device may unlock the locked tactile feedback if control ofthe first object in response to the third control input ends. Inaddition, the display device may unlock the locked tactile feedback evenwhen the third control input is detected at a position exceeding apreset distance threshold from the first region corresponding to thefirst object. Once the locked tactile feedback is unlocked, the displaydevice may generate new tactile feedback according to properties of anobject corresponding to a region where the user control input isdetected or according to properties of a background image.

As described above, the display device may provide continuous tactilefeedback in response to a user control input, thereby achievingconsistent provision of tactile feedback corresponding to an object thatis under control of the user.

As described above, the display device may provide continuous tactilefeedback in response to a user touch gesture, thereby achievinginconsistent provision of an object to be controlled by the user andtactile feedback.

As is apparent from the above description, a display device according tothe present specification may adaptively generate tactile feedbackaccording to a control operation of the user.

Further, according to the present specification, the display device mayprovide preset tactile feedback changed on a per region basis.

Furthermore, according to the present specification, the display devicemay provide continuous tactile feedback in response to a user gesture.

In addition, according to the present specification, the display devicemay lock tactile feedback in response to the user gesture so as toprovide constant tactile feedback while the user controls an object.

It will be apparent to those skilled in the art that variousmodifications and variations can be made in the present specificationwithout departing from the spirit or scope of the specification. Thus,it is intended that the present specification covers the modificationsand variations of this specification provided they come within the scopeof the appended claims and their equivalents.

1. A display device comprising: a display unit configured to display anobject; a tactile feedback unit configured to generate tactile feedback;and a controller configured to: assign first tactile feedback to a firstregion corresponding to a first object and second feedback to a secondregion corresponding to a second object; detect a first control input inthe first region corresponding to the first object, the first controlinput controlling the first object displayed on the display unit;generate the first tactile feedback according to properties of the firstobject; and maintain the first tactile feedback when the first controlinput is moving from the first region to the second region correspondingto the second object while the first object is controlled by the firstcontrol input.
 2. The display device according to claim 1, wherein thefirst control input includes at least one of touch input, gesture input,sound input, and signal input.
 3. The display device according to claim1, wherein the controller is further configured to generate the secondtactile feedback according to properties of the second object inresponse to the first control input when control with respect to thefirst object by the first control input ends in the second region. 4.The display device according to claim 1, wherein the controller isfurther configured to: detect a second control input that is detected inthe second region corresponding to the second object, wherein the secondcontrol input is not in control of the first object; and generate thesecond tactile feedback according to properties of the second object inresponse to the second control input.
 5. The display device according toclaim 1, wherein the second region is a background image displayed onthe display unit.
 6. The display device according to claim 1, whereinthe controller is further configured to display a background image, andwherein generation of the first tactile feedback includes generating thefirst tactile feedback according to texture of the first object andtexture of the background image.
 7. The display device according toclaim 1, wherein the controller is further configured to temporarilygenerate the second tactile feedback different from the first tactilefeedback at a time point when the first control input begins to bedetected in the second region corresponding to the second object.
 8. Thedisplay device according to claim 1, wherein the controller is furtherconfigured to generate the second tactile feedback according toproperties of the second object in response to the first control inputwhen the first control input is detected at a position to which adistance from the first region exceeds a distance threshold.
 9. Thedisplay device according to claim 1, wherein the first object includes aplurality of sub-objects, and wherein the controller is furtherconfigured to generate third tactile feedback according to properties ofa first sub-object and fourth tactile feedback according to propertiesof a second sub-object when the first control input is detected in aregion of the first sub-object and the second sub-object respectively.10. (canceled)
 11. The display device according to claim 1, wherein theproperties of the first object include at least one of texture, color,weight, pattern and brightness of the displayed first object.
 12. Thedisplay device according to claim 1, wherein the controller is furtherconfigured to control the first object in response to the first controlinput such that the first object is controlled based on any one of anX-axis component or a Y-axis component of the first control input. 13.The display device according to claim 1, wherein control with respect tothe first object includes at least one of movement, rotation,enlargement and contraction of the first object.
 14. The display deviceaccording to claim 1, wherein the display device includes at least oneof a smart phone, Personal Digital Assistant (PDA), cellular phone, MP3music player, laptop computer, smart pad, tablet Personal Computer (PC),television, and smart table, which includes the display unit. 15.(canceled)
 16. The display device according to claim 1, wherein thetactile feedback includes non-contact tactile feedback.
 17. The displaydevice according to claim 16, wherein the tactile feedback unit isconfigured to generate tactile feedback using at least one of ultrasonicvibration, acoustic radiation pressure, magnetic field, compressed air,wearable tactile device and remote tactile device.
 18. A method ofcontrolling a display device, the method comprising: assigning firsttactile feedback to a first region corresponding to a first object andsecond feedback to a second region corresponding to a second object;detecting a first control input in the first region corresponding to thefirst object, the first control input controlling the first objectdisplayed on the a display unit; generating the first tactile feedbackaccording to properties of the first object; and maintaining the firsttactile feedback when the first control input is moving from the firstregion to the second region corresponding to the second object while thefirst object is controlled by the first control input.
 19. The methodaccording to claim 18, wherein the first control input includes at leastone of touch input, gesture input, sound input, and signal input. 20.The method according to claim 18, further comprising: generating thesecond tactile feedback according to properties of the second object inresponse to the first control input when control with respect to thefirst object by the first control input ends in the second region. 21.The method according to claim 18, further comprising: detecting a secondcontrol input that is detected in the second region corresponding to thesecond object, wherein the second control input is not in control of thefirst object; and generating the second tactile feedback according toproperties of the second object in response to the second control input.22. The method according to claim 18, wherein the second region is abackground image displayed on the display unit.
 23. The method accordingto claim 18, further comprising: displaying a background image, andwherein generation of the first tactile feedback includes generating thefirst tactile feedback according to texture of the first object andtexture of the background image.
 24. The method according to claim 18,further comprising: generating the second tactile feedback differentfrom the first tactile feedback at a time point when the first controlinput begins to be detected in the second region corresponding to thesecond object.
 25. The method according to claim 18, further comprising:generating the second tactile feedback according to properties of thesecond object in response to the first control input when the firstcontrol input is detected at a position to which a distance from thefirst region exceeds a distance threshold.
 26. The method according toclaim 18, further comprising: generating third tactile feedbackaccording to properties of a first sub-object and fourth tactilefeedback according to properties of a second sub-object when the firstcontrol input is detected in a region of the first sub-object and thesecond sub-object respectively, wherein the first object includes thefirst sub-object and the second sub-object.
 27. The method according toclaim 18, wherein the properties of the first object include at leastone of texture, color, weight, pattern and brightness of the displayedfirst object.
 28. The method according to claim 18, further comprising:controlling the first object in response to the first control input suchthat the first object is controlled based on any one of an X-axiscomponent or a Y-axis component of the first control input.
 29. Themethod according to claim 18, wherein control with respect to the firstobject includes at least one of movement, rotation, enlargement andcontraction of the first object.
 30. The method according to claim 18,wherein the display device includes at least one of a smart phone,Personal Digital Assistant (PDA), cellular phone, MP3 music player,laptop computer, smart pad, tablet Personal Computer (PC), television,and smart table, which includes the display unit.
 31. The methodaccording to claim 18, wherein the tactile feedback includes non-contacttactile feedback.
 32. The method according to claim 31, the tactilefeedback is generated by using at least one of ultrasonic vibration,acoustic radiation pressure, magnetic field, compressed air, a wearabletactile device and a remote tactile device.